Stable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy
نویسنده
چکیده
We extend recent work on the connection between loopy belief propagation and the Bethe free energy. Constrained minimization of the Bethe free energy can be turned into an unconstrained saddle-point problem. Both converging double-loop algorithms and standard loopy belief propagation can be interpreted as attempts to solve this saddle-point problem. Stability analysis then leads us to conclude that stable fixed points of loopy belief propagation must be (local) minima of the Bethe free energy. Perhaps surprisingly, the converse need not be the case: minima can be unstable fixed points. We illustrate this with an example and discuss implications.
منابع مشابه
Stable Fixed Points of Loopy Belief Propagation Are Local Minima of the Bethe Free Energy
We extend recent work on the connection between loopy belief propagation and the Bethe free energy. Constrained minimization of the Bethe free energy can be turned into an unconstrained saddle-point problem. Both converging double-loop algorithms and standard loopy belief propagation can be interpreted as attempts to solve this saddle-point problem. Stability analysis then leads us to conclude ...
متن کاملGraph Zeta Function in the Bethe Free Energy and Loopy Belief Propagation
We propose a new approach to the analysis of Loopy Belief Propagation (LBP) byestablishing a formula that connects the Hessian of the Bethe free energy with theedge zeta function. The formula has a number of theoretical implications on LBP.It is applied to give a sufficient condition that the Hessian of the Bethe free energyis positive definite, which shows non-convexity for...
متن کاملBelief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
We present a novel inference algorithm for arbitrary, binary, undirected graphs. Unlike loopy belief propagation, which iterates fixed point equations, we directly descend on the Bethe free energy. The algorithm consists of two phases, first we update the pairwise probabilities, given the marginal probabili ties at each unit, using an analytic expres sion. Next, we update the marginal proba ...
متن کاملLoopy Belief Propagation, Bethe Free Energy and Graph Zeta Function
We propose a new approach to the theoretical analysis of Loopy Belief Propagation (LBP) and the Bethe free energy (BFE) by establishing a formula to connect LBP and BFE with a graph zeta function. The proposed approach is applicable to a wide class of models including multinomial and Gaussian types. The connection derives a number of new theoretical results on LBP and BFE. This paper focuses tw...
متن کاملOn the Uniqueness of Loopy Belief Propagation Fixed Points
We derive sufficient conditions for the uniqueness of loopy belief propagation fixed points. These conditions depend on both the structure of the graph and the strength of the potentials and naturally extend those for convexity of the Bethe free energy. We compare them with (a strengthened version of) conditions derived elsewhere for pairwise potentials. We discuss possible implications for con...
متن کامل